Generalizability of anaesthesia study populations.

In this issue of the British Journal of Anaesthesia, Sury and colleagues present extensive audit data derived from a survey of anaesthetic departments in the UK, with the general aim of characterizing current National Health Service (NHS) anaesthetic activity. A more specific aim was to provide a context for the release of the upcoming 5th National Audit Project (NAP5) in which the incidence and risk factors for accidental awareness during general anaesthesia in the UK are being quantified. A report last year, based on a similar but preliminary survey, estimated the incidence of awareness in the UK to be about 1:15 000. This finding was based primarily on previous cases made known to senior anaesthetists (trainees were not surveyed) and was much lower than the 1:1000 previously reported in prospective studies, and so may have suffered from detection and reporting bias. The results of NAP5 are thus eagerly awaited. Sury and colleagues provide an exemplar of how to conduct a valid, reliable, and relevant survey. The representativeness, clarity, and completeness of data capture, sampling time and scaling, and high response and return rates support the accuracy and precision of the results. Indeed, each of the large-scale surveys conducted as part of the ongoing NAP series have been conducted in accordance with recommended standards. 4 Surveys are generally a less expensive and convenient method of characterizing a clinical setting, practices, and outcomes. They do however require substantial planning. The target population (sample frame), measured variables, and types of associations being investigated should be specific and planned in advance. Key goals when implementing a survey include unbiased sampling and maximizing the response rate to avoid misleading results. Specific sampling methods such as random or sequential sampling are used to minimize bias. Perhaps the most important necessity for a valid study is the identification of a complete sampling frame, which is a listing of all individuals constituting the population of interest. In the current case, this is the anaesthetic services for the whole UK NHS. A response bias will occur if those who respond to a survey do not reflect the greater population—perhaps because of greater interest in the topic under study, or respondents may be more likely to have extreme views or experiences because of greater interest to the survey topic. For example, surgical audit projects may well over-estimate outcomes when compared with noncompliant centres. Therefore, strategies to reduce the proportion of non-responders to ,30% (ideally ,10%) are highly recommended. Clearly, increased survey participation is likely to limit this gap, and so minimize sources of selection bias that could invalidate the estimation of population characteristics. The determination of the size or duration of the sampling is a balance between gaining enough data in order to have precise estimates against the added cost and complexity of a larger survey. Sury and colleagues chose to collect their sample data over a 1 week period, and then scaled these data to represent a 1 yr period in the UK. One could quibble over aspects of this approach, but alternative approaches are not likely to be any more accurate within the resources that were available. The time of year that this survey was conducted may be a factor however, in that the July effect might have exposed hospitalized patients to more inexperienced junior doctors. 7 However, these findings are almost certainly more representative of contemporary anaesthetic practice in the UK when compared with earlier surveys of practice. So why are the data presented by Sury and colleagues relevant to readers of this journal, including those working outside of the UK? NAP5 will provide the numerator for accidental awareness during general anaesthesia in the UK, and the denominator data provided by the current survey can be used to estimate risk, and also provide the descriptive detail needed to understand the clinical context in which such awareness cases occur. The environment or context of any study setting is important. Anaesthetists practicing in the UK, the rest of Europe, and indeed all over the world, including as far away as Australia and New Zealand, must consider whether or not the NAP5 findings can be applied to their practice. Extrapolating the findings of any study to new environments requires an understanding of sampling theory and survey methodology. Sampling is the process by which inference is made to a target population from a sample. Clinicians often want to apply the results of a specific survey to a population which was not part of the sampling frame (i.e. outside of the UK). That is, will the findings of NAP5 be generalizable—does it have external validity? Sury and colleagues provide the necessary information for readers of this journal to make such a judgement. Indeed, such extrapolation is often needed when interpreting results of a clinical trial, and all such extrapolations require very careful consideration. 9 Editorial BJA